Accelerating the assessment agenda: thinking outside the black box

Over the last 10 years, learning and teaching in higher education have benefited from advances in social constructivist and situated learning research (Laurillard, 1993). In contrast, assessment has remained largely transmission orientated in both conception and in practice (see Knight & Yorke, 2003). This paper examines a number of recent developments, which exhibit innovation in electronic assessment developed at the UK's Open University. This paper argues for the development of new forms of e-assessment where the main driver is that of sound pedagogy rather than state of the art technological know-how and where open source products can move the field forward

Computer assisted formative assessment: supporting students to become more reflective learners

e-Assessment is being advocated in the UK as our way of introducing a more personalised learning agenda throughout the Higher Education sector. This paper discusses the findings from two projects where formative e-assessment has contributed to students taking more control of their own learning. One study set out to provide further insights into the role of electronic formative assessment and to point the way forward to new assessment practices, capitalising on a range of open source tools. The guiding vision was to pilot a series of formative assessments which have the potential to help shape learners as independent thinkers, making their own judgements and decisions about their learning in partnership with their peers and tutors. Other work consisted of evaluating a series of formative assessments given to Philosophy students. Lessons have been learned about the type of feedback that instructors and students think will be most useful and how using theis type of application promotes self reflection. The research reported here starts to illustrate how technology can be adapted to become more 'fit for pedagogical purpose?. The feedback offered by these systems encourages learner metacognition and aims to empower students to reflect and become independent thinkers. This approach sits well within a constructivist paradigm which has often been less well served in the past through formal summative assessment which is not an integral part of the knowledge construction process

Towards a model for evaluating student learning via e-assessment

The need for practical tools to assess student learning at the course level is becoming a more pressing goal for all academic institutions. This is because learning assessment tools which monitor both student performance and conceptual change events that lead to improved learning ultimately provide the basis for the subsequent assessments of programs and institutions. In performing effectively in this capacity, a viable and efficient assessment tool at the course level possesses the following characteristics; 1) the ability to be integrated effectively within the existing course structure, 2) the ability to generate quantitative, measurable results, and 3) the ability to provide timely feedback. This paper proposes a model for assessing student learning at the course level which utilizes, in part, online assessment methods (eAssessments) to achieve these characteristics. More specifically, the model provides a description of how assessment may be embedded into an existing course and illustrates the utilization of online pre/post-tests and knowledge surveys as a source of assessment data. The data analysis, based in part upon Bloom's revised taxonomy, is then discussed together with how the results are used to determine the level of learning achieved. The paper concludes with a proposal for an experiment wherein the model is tested to determine its ability to detect changes in student learning originating from the implementation of a pedagogical strategy such as online tutoring

Benchmarking assessment: breaking down barriers and building institutional understanding

Benchmarking offers a comprehensive way of measuring current practice in an institution; whilst also gauging achievement against external sources. Although e-learning has been benchmarked with a number of universities in the UK and abroad no one to date has tackled the area of assessment; which is now becoming of more concern with the advent of e-assessment. This paper describes the construction of a set of benchmarking measures/indicators and the outcome of early pilots which combine a survey instrument and semi-structured interview methodologies. The findings suggest that building a comprehensive and robust core of benchmark measures can have great utility and value to institutions; not just in external benchmarking but also in internal reviews. It can also assist with setting baselines, exploring the student experience, providing staff with data meaningful to their role and professional development together with supporting a continuous improvement trajectory

Piloting a new approach: Making use of technology to present a distance learning computer science course

Computer‐Mediated Communication (CMC) systems have been described and evaluated in a number of ways by different researchers in the field. This paper proposes that computer conferencing systems should be designed to encourage students to participate in three dimensions previously treated by separate researchers. These can be summarized as a knowledge dimension, a social dimension and a motivational dimension. This paper reports on how one particular conference, that of M205‐STILE, was constructed to take account of these dimensions and to facilitate students’ computer‐supported cooperative learning

E-assessment: how can we support tutors with their marking of electronically submitted assignments?

One of the problems with tutor feedback to students is that a balanced combination of socio emotive and cognitive support is required from the teaching staff, and the feedback needs to be relevant to the assigned grade. Is it possible to capitalise on technology to build training systems for tutors in Higher Education, that will support them with their feedback to students, and which will encourage their students to become more reflective learners? One approach to this question was to build an electronic tool to support tutors in the feedback process. Our work has involved building an open-source mentoring tool for tutors, known as Open Mentor. This tool analyses and displays the different types of comments provided by the tutor as feedback to the student. It then provides reflective comments to the tutor about their use of feedback. Open Mentor was not designed for use at institutional level, but to give teaching staff a tool that can be used in training and also later as personal support that will enable individual tutors to track their use of feedback to students

Is it sustainable? A comparison of student and tutor online time across three distance‐learning courses

Distance‐learning course providers world‐wide are looking to new technology to enhance their course presentations. One of the areas they are beginning to assess is that of integrating Computer Mediated Communication (CMC) into their courses. An important factor to be borne in mind with the adoption of CMC is the amount of time required to be an active participant. It is important therefore to investigate how much time tutors and students spend online, and if this is sustainable in the longer term. This paper discusses tutor and student involvement in the Open University M205 ‐ STILE project, and compares these findings with the time spent online by students and tutors on two other Open University courses. Our aim was to see if our findings were atypical. The comparisons are discussed, and recommendations are made about how future online courses can benefit from the findings of this study

Assessment benchmarking: accumulating and accelerating institutional know - how for best practice

Benchmarking offers a comprehensive way of measuring current practice in an institution; whilst also gauging achievement against external sources. Although e-learning has been benchmarked with a number of universities in the UK and abroad no one to date has tackled the area of assessment; which is now becoming of more concern with the advent of e-assessment. This paper describes the construction of a set of benchmarking measures/indicators and the outcome of early pilots which combine data from a survey instrument of these measures with semi-structured interviews. The findings indicate that the benchmark measures this project has identified can form a solid foundation for benchmarking and that a mixed methods approach built around thisa comprehensive and robust core of benchmark measures can have value to institutions; not just in external benchmarking but also in internal reviews. It can also assist with setting baselines, exploring the student experience, providing staff with data meaningful to their role and professional development together with supporting a continuous improvement trajectory